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SUMMARY 

Results of calibration tests conducted by the Bureau of Standards on three 
R-20 Rubidium Frequency Standards submitted by Varian Associates of Palo 
Alto, California are analyzed. The technique of least squares is used to calcu- 
late "best fit" zero, first, second, and third degree polynomials which describe 
as a function of time the variation in relative frequency of each standard from a 
reference — the United States Frequency Standard. A pooled sample of devia- 
tions of the daily readings from the daily averages is compared to a normal dis- 
tribution having the same mean and standard deviation. This is done for each 
unit, hi addition, the residuals of the "best fit" straight lines for each frequency 
standard are compared to a normal distribution. The deviations of the daily 
readings from the daily averages as well as the residuals of the "best fit" 
straight lines show fairly good agreement with a normal distribution. 

Using the expressions for the "best fit" straight lines, the drifts in rela- 
tive frequency over a period of one year are: -(13.6)10 ~ n , +(34.1)10 _n , and 
+(1.7)10- 11 . 

The standard deviations of the polynomials are determined for each fre- 
quency unit and plotted as a function of time. It is shown that the straight line is a 
more "stable predictor” of Rubidium Frequency Standard performance than a 
higher degree polynomial in the sense that the standard deviation of the straight 
line remains within an arbitrary tolerance of 1 part in 10 11 for a much longer 
period of time (weH over a year) than does a higher degree polynomial. 

The 90% and 95% tolerance limits for the deviations of the daily readings 
from the "best fit" straight lines are also shown for each frequency standard. 
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ANALYSIS OF THE CALIBRATION TESTS 
FOR THE APOLLO VARIAN RUBIDIUM 
FREQUENCY STANDARDS 


INTRODUCTION 

Results of calibration tests, performed by the National Bureau of Standards, 
on three R-20 Rubidium Frequency Standards (serial 100, 106, and 107) submitted 
by Varian Associates of Palo Alto, California, have been analyzed. Data on serial 
100 covered the period from 22 July 1965 to 15 November 1965; data on serial 106 
covered the period from 1 September 1965 to 6 December 1965; and, data on 
serial 107 covered the period from 23 August 1965 to 15 November 1965. 

An explanation of the test procedure is given below as well as a discussion of 
the least squares polynomial fitting to the data, a statistical analysis of the devi- 
ations of the daily readings from the daily averages, and the determination of the 
standard deviation of the polynomial. A section is devoted to a discussion of the 
least squares technique and the determination of the covariance matrix of the co- 
efficients. 

The most salient feature of the analysis is that a straight line fit to the data 
provides us with a more ’’stable predictor" of frequency standard performance 
than does a higher degree polynomial fit. 


TEST PROCEDURE 

A simplified block diagram of the Rubidium Frequency Standard Calibration 
Test is shown in Figure 1. Letting t = time in seconds, A 0 = amplitude of the 
signal from the R-20 (volts) , and <£ 0 (t) = phase angle of the R-20 (radians) , we 
can write an expression for the signal from the R-20 oscillator, V 0 (t), as: 


V Q (t) = \ cos W*)) (1) 

where 


0 o (t) = 2 t 7 f 0 1 + cp 0 (t) (radians) 

f = R-20 nominal frequency = 5MHz 

(P o (t) =: variation of R-20 phase angle due to random noise, 
frequency offset, and drift (radians) 

1 



The signal from the R-20 Test Oscillator is fed into a frequency multiplier 
(K) and the output signal, V x (t) , is: 

Vj(t) = A 0 cos [K0 O (t)] . (2) 

Vj(t) is then fed into a phase detector. A signal, V 2 (t) , from the USFS 
(United States Frequency Standard - Cesium) with a nominal frequency f g = 
9192.631770 MHz (Reference 1), amplitude A, (volts), and phase angle & (t) 
(radians), is also fed into the phase detector, v (t) can be represented by: 

V 2 (t) = A 2 cos [0 s (t)] (3) 

where 


0 s (t) = 2wf s t + cp s (t) (radians) 

cp s (t) = variation in phase angle of the USFS due to random 
noise (radians) 

The output difference signal from the phase detector, V(t) , then becomes: 
V(t) = Acos[27r(f s -Kf 0 )t + (cp s (t)-K<P 0 (t))] (4) 

where 


A = amplitude (volts) 

K - multiplying factor 

Letting cp 0 (0) and <P S (0) represent respectively the values of <P 0 (t) and 
cp s ( t ) at time t = 0 , the change in phase angle , Acp , at the output of the detector , 
after a time t = T seconds may be written: 

A<p = cp s (T) - Kcp 0 (T) - [cp s (0) - Kcp 0 (0) ] (5) 

or 


Acp = K Acp 0 

where 


and 


cp s (T) - cp s (0) is small 
Acp 0 = <P Q (0) -«p 0 (T) 


2 


The change in phase angle, Acp 0 , due to random noise, frequency offset, and 
frequency drift in the R-20 over an interval of T = 200 seconds (a time interval 
corresponding to 10 cycles of the difference frequency), was measured and 
divided by T to obtain: 



Each day approximately 6 to 12 observations as given by equation (6) were 
made. 

The values as recorded in references 2, 3, and 4 are expressed in (Af 0 /f 0 ) 
10 10 units. In the same references, it is stated that the absolute accuracy of the 
USFC is presently believed to be ±6 x 10 -12 . No indication of the meaning of this 
number is given. 

ANALYSIS AND DISCUSSION 

In the following paragraphs we will discuss the analysis techniques and the 
results obtained in this report. 

Least Squares Polynomial Fitting to Daily Averages 


The daily averages 1 were calculated using all of the daily observations ex- 
cept in those instances where an observation varied significantly from the others. 
In these cases the particular observation was thrown out and the average com- 
puted using the remaining observations. Figures 2, 3, 4, and 5 show those ob- 
servations which were excluded. It should be noted that these observations were 
not thrown out by any fixed criterion but rather by an "eye-smoothing" process. 

A least squares polynomial was then fitted to the observed daily averages 
for each Rubidium Standard. These results may be seen in Figures 6 through 11 
inclusive. Figures 6, 8, and 10 which refer to serials 100, 106, and 107 respec- 
tively contain plots of the daily averages versus time (in days) as well as the 
"best fit" (in the sense of least squares) polynomials of degree 0, 1, 2, and 3 for 
these data points. Figures 7,9, and 11 also pertain to serials 100, 106, and 107 
respectively, but are for extended periods of time - up to one year. In each fig- 
ure the equation of the "best fit" polynomial is indicated as well as the standard 


^The daily averages were used instead of the daily readings since there were no recordings of 
the time of the daily readings. 
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deviation of fit, 17. 2 It should be noted that two daily averages were excluded in 
order to "smooth out" the data (see Figures 8 and 10). 

Using the expressions for the "best fit" straight lines, the drifts in relative 
frequency over a period of one year (T = 365) are: -(13.6) 10" 11 for serial 100; 
+(34.1) 10" 11 for serial 106; and, +(1.7)10“ 11 for serial 107. 


Statistical Analysis of Daily Variation and Variation of the 

Residuals of the "Best Fit" Straight Line 

A pooled sample of deviations of the daily readings from the daily averages 
was taken for each frequency standard (106 for serial 100, 110 for serial 106, 
and 107 for serial 107) and the standard deviation calculated. Figure 12 shows 
the histograms for the samples compared to normal distributions having the 
same mean (zero) and standard deviation. 

Figures 13, 14, and 15 show the histograms of the residuals or the devia- 
tions of the daily averages from the "best fit" straight lines. Again the 
histograms are compared with normal distributions having the same mean and 
standard deviation. For a normal distribution the expected number of observa- 
tions falling outside ±1.64 standard deviations (these limits correspond to a 
probability of 90%) is 4.5 for a sample size of 45 (serial 100), 3.8 for a sample 
size of 38 (serial 106), and 2.9 for a sample size of 29 (serial 107). The ob- 
served numbers which fell outside these limits were 5, 3, and 2 for serial 100, 
106, and 107, respectively. It does not seem unreasonable, therefore, to assume 
a normal distribution for the residuals. 

Figures 16, 17, and 18 show the 90% and 95% tolerance intervals for the 
deviations of the daily readings from the "best fit" straight lines for the three 
units. In calculating these limits a normal distribution was assumed for the 
residuals. The standard deviation of the daily readings from the straight line 
was calculated using the following relationship: 

S = /t ? 2 + Sj ^ 


^ The standard deviation of fit 77 is computed as: 

V = [y. - P (x.)] 2 where n = number of data points, 


fi - k + 1, k * degree of the polynomial, P (x ; ) = value of the “best fit” polynomial at the point x. 
(i =1,2,... n), y. = observed value corresponding to x ; (see Reference 5). 
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where tj = standard deviation of fit and s d is the standard deviation of the 
pooled sample. 3 

The Standard Deviation of the Least Squares Polynomial 

As a Function of Time 4 

Having calculated the coefficients of a least squares polynomial to a set of 
experimental data, one may ask, "How good is the polynomial in predicting the 
performance of the Rubidium Standards?" Or, phrasing the question another 
way, "What is the spread of values that the polynomial can take as a function of 
time based upon the finite sample of data points ? " In order to answer this 
question, it will be sufficient to write an expression for the standard deviation 
of the polynomial at a given instant of time. 

Let the form of the least squares polynomial be 

y = P(x) = a Q + ajX + a 2 x 2 + ... + (k > 0) (8) 

When x = Xj , y will have the value y . given by 

y, = P( x j) = a 0 + a i x j + ••• + a k x j ( 9 > 

Considering the coefficients as random variables we can write an expression 
for the standard deviation of y when x = x. as follows: 



( 10 ) 


where a 2 = a is the variance of a m and o- is the covariance be- 

, , a a a m m a m a m+n 

tween a and a , . 

m m T n 


^This relationship holds if it is assumed that the statistics do not change with time and that the 
deviations of the daily readings from the "best fit" straight line are uncorrelated. 

rather detailed description of the known “Method of Least Squares" is presented for the 
convenience of the reader. 
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As an example, the standard deviation of y. for a second degree polynomial 
when x = Xj can be written as 


i° 


X 2 cr 2 


+ *7 CT ; 


2x.a 


o“i 


2 XT Cr 

) a 


0 2 


2x?£7 

J 8 


12 


(ID 


In order to determine the variances and covariances in (10) we will briefly 
describe the least squares technique using matrix notation in order to simplify 
the algebra. 


Brief Discussion of the Least Squares Technique and the Determination 
of the Covariance Matrix of the Coefficients 


Reference (5) contains a discussion of the generalized case of the method 
of least squares and, in addition, includes a procedure for calculating the un- 
certainty associated with each quantity determined by the least squares method. 

In this report the quantities are the coefficients of the "best fit" polynomials 
and the uncertainty is the standard deviation of each coefficient. We will now 
briefly describe the least squares method and determine the covariance matrix 
of the coefficients from which the uncertainties associated with the coefficients 
as well as the covariances between the coefficients can be obtained. 

Let the data be represented by the set of n points (x »y t )» (^’i^)’ • • • » 

(x ,y ) where it is assumed that the errors are in the ordinates y . only and let 

the form of the polynomial be as indicated in (8) with n > k. The least squares 

method assumes that the "best" set of values for the coefficients a. is that set 

1 

which minimizes the sum S of the squares of the deviations ( v.’s) of the observed 
quantities (y. ’s) from the values of the polynomial (y i ’s). This can be expressed 
mathematically as 


n 



i=l 



( 12 ) 


In order to minimize S it is necessary to take the partial derivatives of S 
with respect to each of the coefficients and set these derivatives equal to zero. 
When this is done k + 1 "normal" equations in k + 1 unknowns result. The set 
of coefficients is then the set of roots which satisfy the k + 1 simultaneous 
linear equations. The normal equations are given by: 
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The set of equations in (13) can be solved by standard techniques. For ease 
of notation let us now use matrices. First, we can write down n "observational” 
equations which express each observation as a function of the least squares 
polynomial evaluated at the point x. and the residual v. (j = 1, 2, . . . n) . 

yi = a o + a i x i + ••• + + v i 


(14) 


Letting " 


y n 

= a o 

+ a i X n + + 

V . 
n 


yi 


1 X* ... X* 


a o 

y 2 


1 X 2 X* ... X* 


a i 


A = 


x = 


• 


• 


• 

• 


• 


• 



• 


• • 

y n 

i 

! 

1 X X 2 . . . x k 

n n n 


*k 


A T denotes the tronspose of A; A* 1 denotes the inverse of A; 8( V) denotes the expectation of 
a random matrix V. 8(W T ) denotes the covariance matrix of a random matrix V with zero mean. 
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and 


the equations in (14) become 

Y = AX + V 

The normal equations in (13) may be written as 

(A t A) x = a t y 


(15) 


(16) 


Solving the matrix equation (16) for X we obtain the "best" estimate for the 
coefficients 


X=(A t A)“ 1 A t Y (17) 

In order to find the covariance matrix of the coefficients we must find the 
expected value of (X-X) (X-X) T * Substituting (15) into (17) one obtains after 
some manipulation 

£(X - X)(X - X) T = (A T A) -1 A t £(YV t ) A^A)' 1 (18) 

If we assume the residuals v. ( i = 1,2, . . . n) to be uncorrelated random 
variables each with variance r / 2 (the square of the standard deviation of fit, 
previously defined) we have 


8(V\ T ) = rj 2 l 


(19) 


where I is the n x n identity matrix. 

Substituting (19) into (18) we have the required covariance matrix of the 
coefficients 

8(X - X)(X - X) T = if(jFAy l (20) 
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Letting P denote the covariance matrix of the coefficients, and Q the inverse 
of A t A, we can write 


P = 


<J 


0 

cr- 

a o a i a i 


12 


( 21 ) 


cr a 

a 0 a k a l a k 



q n 

<*12 

'*• q i,k+l 

q 21 

q 22 

*'■ q 2.k+l 

^ + 1, 1 


‘^k + l.k + l 


As an example, the covariance between a x and a 2 would be given by q 23 rf 
and the variance of a 3 would be given by q 44 17 2 . 

The variances and covariances of the coefficients were calculated for each 
degree polynomial (zero, first, second, and third) and for each frequency stand- 
ard. They were then substituted into (10). The standard deviation of the poly- 
nomial was then plotted as a function of x. and can be seen in Figures 19, 20, 
and 21 . 


It is evident from these figures that the uncertainty (standard deviation) of 
the second and third degree polynomials increases sharply just after the period 
of observation and that the standard deviation of the straight line stays within an 
arbitrary tolerance of 1 part in 10 11 for a much longer period of time (over a 
year) than the standard deviation of the higher degree polynomial. 


Table I gives the standard deviation of the coefficients of the least squares 
polynomials, and Table II gives the length of time in days during which the 
standard deviation of the polynomial is less than 1 part in 10 11 . 
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CONCLUSIONS 


(1) A "best fit" straight line serves as a more "stable predictor" of Rubid- 
ium Frequency Standard performance than a higher degree polynomial in the 
sense that the standard deviation of the straight line remains within an arbitrary 
tolerance of 1 part in 10 u for a longer period of time (over 1 year) than the stand- 
ard deviation of a higher degree polynomial. 

(2) The "best fit" straight lines for Rubidium Frequency Standards R-20 
Serials 100, 106, and 107 give the following drifts over a period of one year: 
-(13.6)10-ii (serial 100); +(34.1)10-n (serial 106), and +(1.7)10-n (serial 107). 

(3) For periods of time comparable to those over which the observations 
were made on Rubidium Frequency Standards R-20 Serials 100, 106, and 107, 
and under the assumptions indicated, there are 90% and 95% probabilities that 
the daily readings will fall within the boundaries as shown in Figures 16, 17, 
and 18. 
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Table I 

Standard Deviations* of the Coefficients of the 
Least Squares Polynomials 

General Form of the Least Squares Polynomial (see Figures 6, 8, and 10). 

) 10 10 = a o + a i T + • • + a k T k 

where k is the degree of the polynomial and T is time (in days). 



k 

cra o 

era j 

cra 2 

aa 3 

Serial 100 

0 

.14600 





1 

.01735 

.2775 x 10- 3 




2 

.02944 

1.228 x 10~ 3 

.9842 x 10- 5 



3 

.04194 

3.083 x 10~ 3 

6.243 X10- 5 

3.541 x 10- 7 

Serial 106 

0 

.29090 





1 

.01703 

.3004 X 10' 3 




2 

.01408 

.7491 x 10~ 3 

.7929 x 10- s 



3 

.01436 

1.533 x 10- 3 

3.970 x 10- 5 

2.817 x 10- 7 

Serial 107 

0 

.04170 





1 

.01299 

.2810 x lO" 3 




2 

.01543 

1.024 x lO" 3 

1.292 x lO' 5 



3 

.01914 

2.560 x lO" 3 

7.780 x 10- 5 

6.400 x 10- 7 


All values in the body of the table are in parts in 10 M . 
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Table II 

Length of Time in Days During which the Standard Deviation 
of the Polynomial* is Less Than 1 Part in 10 11 



Linear 

Quadratic 

Cubic 

Serial 100 

411 

165 

133 

Serial 106 

380 

161 

125 

Serial 107 

393 

129 

99 


The standard deviation of a k th degree polynomial when x = x- ( is 
given by the following relationship: 


o_ 

Vi 


k~*rr 


kEL 

m*0 n *0 


v 2m +n 

i 


a a , 
m m +rt 



where a?~a a a is the variance of a m and C7 a fl is the covari 
m a m u m m m +n 

ance between a m arida m+n . 
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Figure 7-“Best Fit* Least Squares Polynomials of Degree 0, 1, 2, and 3 for 
Varian Rubidium Standard R-20 Serial 100 (extended time scale) 
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Figure 9-“Best Fit* Least Squares Polynomials of Degree 0, 1, 2, and 3 for 
Varian Rubidium Standard R-20 Serial 106 (extended time scale) 
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Figure 11-“Best Fit” Least Squares Polynomials of Degree 0, 1, 2, and 3 for 
Varian Rubidium Standard R-20 Serial 107 (extended time scale) 
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Note: 

fadings for Several Days were Pooled 
to Increase the Sample Size 
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Figure 15-Histogram of Residuals of “Best Fit" Straight Line Compared to a Normal Distribution - Serial 107 
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Best Fit” Straight Line - Serial 107 
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Figure 19-Standard Deviation of the Polynomial VS Time - Serial 100 
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Figure 20-Standard Deviation of the Poly 
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Figure 21 -Standard Deviation of the Polynomial VS Time - Serial 107 
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